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Abstract 

We consider the problem of recovering a partially sparse solution of an underdetermined system of linear equations by 
minimizing the £i-norm of the part of the solution vector which is known to be sparse. Such a problem is closely related to a 
classical problem in Compressed Sensing where the £i-norm of the whole solution vector is minimized. We introduce analogues 
of restricted isometry and null space properties for the recovery of partially sparse vectors and show that these new properties are 
implied by their original counterparts. We show also how to extend recovery under noisy measurements to the partially sparse 
case. 

Index Terms 

Partial sparse recovery, compressed sensing, ^i -minimization. Sparse quadratic polynomial interpolation. 

I. Introduction 

IN Compressed Sensing one is interested in recovering a sparse solution x S R^ of an underdetermined system of the form 
y = Ax, given a vector y € R*^ and a matrix A € R'^^^ with far fewer rows than columns {k <^ N). A direct approach is 
to minimize the number of non-zero components of x, i.e., the (.Q-norm of x (which is defined as ||m||o — \{i ■ Ui ^ 0}| but, 
strictly speaking, is not a norm), 

min||a;||o s.t. Ax = y. (1) 

Since (HJ is known to be NP-Hard, a tractable approximation is commonly considered which is obtained by substituting the 
non-convex io-norm by a convex approximation. Recent results indicate that the ^i-norm can serve as such an approximation 
(see m for a survey on some of this material). Hence ([T) is replaced by the following optimization problem 



s.t. 



Ax 



y- 



(2) 



Note that (|2]i is equivalent to a linear program and thus is much easier to solve than ([T]l. 

In this paper we consider the case (see ||2l, ||3|, ||4]) when it is known a priori that the solution vector consists of two parts, 
one of which is expected to be dense, in other words we have x = (j:i,a;2), where xi £ R^^'' is sparse and X2 G R'^ is 
possibly dense. A natural generalization of problem (ID to this setting of partially sparse recovery is given by 



xilli s.t. A1X1 + A2X2 ^ y, 



(3) 



where A = (^1,^2), Ai € R''^(^~''), and A2 G R'^^''. We will refer to this setting as partially sparse recovery of size 
N — r. One of the key applications of partially sparse recovery is image reconstruction [2] but they also arise naturally in 
sparse Hessian recovery ||5]. 

Vaswani and Lu |2| gave a first sufficient condition for partially sparse recovery. Later, Friedlander et al. [3 j proposed a 
weaker sufficient condition and covered the extension to the noisy case. After obtaining our results we were directed to the 
work of Jacques [4| who addressed the noisy case, deriving another sufficient condition for partially sparse recovery. His 
conditions guarantee the same recovery as ours but, as far as we can tell, are not the simple extensions of the NSP and RIP 
properties. The conditions in [12], lEI. iSl are somewhat weaker than the known restricted isometry property for general sparse 
recovery, which is natural since the case of partial sparsity can be considered as a case of general sparsity where part of the 
support of the solution is known in advance. 

The contribution of our paper is to introduce the analogues of restricted isometry and null space properties for the case of 
partial sparsity. We prove that these new properties are sufficient for partially sparse recovery (including the noisy case) and 
are implied by the original conditions of fully sparse recovery. We show that it is possible to guarantee recovery of a partially 
sparse signal using Gaussian random matrices with the number of measurements an order smaller than the one necessary for 
general recovery. 
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A. Notation 

We will use the following notation in this paper. [N] denotes the set of integers {1, . . . , N}, and [N]^^'> denotes the set of 
all subsets of [N] of cardinality s < A^. If A is a matrix, then by Af{A) and Tl{A) we denote the null and range spaces of A, 
respectively. We say that a vector x is s— sparse if at most s components of x are non-zero. This is also denoted by ||a;||o < s. 

Given w e M^ and 5 G [N], vs € M^ denotes a vector defined by {vs)i ^ Vi, i e S and {vs)i = 0, i ^ S. 

II. Sparse recovery in compressed sensing 

One of the main questions addressed by Compressed Sensing is under what conditions on the matrix A can every sparse 
vector X be recovered by solving problem (|2) given A and the right hand side y = Ax. The next definition is a well known 
characterization of such matrices (see, e.g., [6J, [7]). 

Definition 2.1 (Null Space Property): The matrix A g R*^^^ is said to satisfy the Null Space Property (NSP) of order s if, 
for every v £ A/'(A) \ {0} and for every S € [N]'^^\ one has 

hsWi < l\\v\\i. (4) 

It is well known that NSP is a necessary and sufficient condition for the recovery of an s-sparse vector x (see (S)). 

Theorem 2.1: The matrix A satisfies the Null Space Property of order s if and only if, for every s— sparse vector x, 
problem ^ with y ~ Ax has an unique solution and it is given hy x = x. 

It is difficult to analyze whether NSP is satisfied. On the other hand, the Restricted Isometry Property (RIP), introduced 
in ||9l, is considerably more useful and insightful, although it provides only sufficient conditions for recovery with (|2]i. We 
present below the definition of the RIP Constant. 

Definition 2.2 (Restricted Isometry Property Constant): One says that (5s > is the Restricted Isometry Property Constant, 
or RIP constant, of order s of the matrix A e M*^^^ if (5^ is the smallest positive real number such that: 

{1^5,)\\x\\l < \\Ax\\l < {l + Ss)\\x\\l (5) 

for every s— sparse vector x. 

The following theorem (see, e.g., [lOJ ) provides a useful sufficient condition for successful recovery by (|2). 

Theorem 2.2: |10| Let A G R''^^ and 2s < k. If 52s <V^~ 1, where 52s is the RIP constant of A of order 2s, then, for 
every s— sparse vector x, problem dU with y = Ax has an unique solution and it is given hy x = x. 

It is known that RIP is satisfied with some probability if the entries of the matrix are randomly generated (see, e.g., [11]) 
according to some distribution such as a sub-Gaussian. However, it is in general computationally hard to check whether it is 
satisfied by a certain realization matrix [12], and it is still an open problem to find such matrices deterministically when the 
underlying system is highly underdetermined (see llT3l ). 

III. Partial sparse recovery 

In this section we consider the following extension of the NSP to the case of partially sparse recovery. 
Definition 3.1 (Partial Null Space Property): We say that A ~ {Ai,A2) satisfies the Null Space Property (NSP) of order 
s — r for partially sparse recovery of size N — r with r < s if A2 is full column rank (J\f{A2) — {0}) and for every 

vi e R^"'' \ {0} such that AiVi £ n{A2) and every S E [N - r] (*-''), we have 

ll(«i)s||i < ^ll^illi. (6) 

Note that when r = 0, the partial NSP naturally reduces to the NSP in Definition 12.11 Wang and Yin |[T4ll have suggested a 
stronger NSP adapted to a setting where it is not known the location of the partial support. 

The new property is a necessary and sufficient condition for any solution of (O with y = Ax to satisfy a; = a; if xi is 
appropriately sparse. 

Theorem 3.1: The matrix A — (^1,^2) satisfies the Null Space Property of order s — r for Partially Sparse Recovery of 
size iV — r if and only if for every x — (xi,X2) such that xi G R^"'" is (s — r)— sparse and X2 S R'^, problem ^ with 
y — Ax has an unique solution and it is given by (a;i,a;2) = {xi,X2). 

Proof: The proof follows the steps of the proof of ||8] Theorem 2.3] with appropriate modifications. Let us assume 
first that for any vector (a;i,a;2) € R^, where xi is an (s — r)— sparse vector and X2 € R*^, the minimizer (a;i,a;2) of 
||a;i||i subject to Aixi + A2X2 = Ax satisfies xi = xi. Consider any t^i 7^ such that AiVi £ TZ{A2). Then consider 
minimizing |lxi||i subject to AiXi + A2X2 = Ai{vi)s + A2V2 for any V2 G R' and for any S £ [N — r]*^''^'^ By the 
assumption, the corresponding minimizer (a;i,a;2) satisfies xi = {vi)s. Since AiVi £ TZ{A2), there exists U2 such that 
Ai{-{vi)so) + A2U2 = Ai{vi)s + A2V2. As -(wi)s<= 7^ {^1)3, (-(wi)s<=,"2) is not the minimizer of ||xi||i subject to 
Aixi + A2X2 = Ai{vi)s + A2V2, hence, ||(wi)sc||i > ||(wi)s||i and dill holds. 

Let us now assume that A satisfies the NSP of order s — r for partially sparse recovery of size N — r (Definition 13.11) . 
Then, given a vector (xi, X2) G R^, where xi is (s — r)— sparse and X2 £ M.^, and a vector (wi, ■U2) G K^ with ui 7^ xi and 
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satisfying AiUi + A2U2 = AiXi + A2X2, consider {vi, V2) — {{xi — ui), {x2 — U2)) G J^{A), which imphes AiVi G TZ{A2) 
and vi 7^ 0. Thus, setting S to be the support of x, one has that 

ll^illi < ||Si - (ui)s||i + ||(mi)s||i 

= ||(«l)5||l + ||(^*l)s||l<||(t'l)5Hll + ll("l)5|ll 

= II - {ui)s-\\i + ||(wi)s||i = ll"i||i, 

(the strict inequality coming from (|6]l), guaranteeing that all solutions (xi,X2) of ^ with y = Ax satisfy xi = xi. 

It remains to note that X2 = X2 is uniquely determined by solving A2X2 = y — Aixi if and only if A2 is full column rank. 

■ 

We now define an extension of the RIP to the partially sparse recovery setting. For this purpose, let A — (^1,^2) be as 
considered above, under the assumption that A2 has full column rank. Let 

V = I - A2 {Aj A2y' Aj (7) 

be the matrix of the orthogonal projection from M^ onto 7?, (^2) ■ Then, the problem of recovering (a;i,a;2), where xi is 
an (s — r)— sparse vector satisfying AiXi + A2X2 ~ y, can be stated as the problem of recovering an (s — r)— sparse vector 
xi — xi satisfying (VAi) xi = Vy and then recovering X2 ~ X2 satisfying ^22^2 ~ V ^ AiXi. The solution of the resulting 
linear system in the second step exists and is unique given that A2 has full column rank and {VAi)xi — Vy. Note that the 
first step is now reduced to the classical setting of Compressed Sensing. This motivates the following definition of RIP for 
partially sparse recovery. 

Definition 3.2 (Partial RIP): We say that 6l_,. > is the Partial Restricted Isometry Property Constant of order s — r of 
the matrix A = {Ai,A2) G M.'^^^, for recovery of size N ~ r with r < s, if A2 is full column rank and Sl_j. is the RIP 
constant of order s — r (see Definition 12.21 ) of the matrix VAi, where V is given by (|7]l. 

Again, when r — the Partial RIP reduces to the RIP of Definition 12.21 We also note that, given a matrix A = {Ai,A2) E 
^kxN ^jj-jj Partial RIP constant (52(s-r) of order 2(s — r) for recovery of size N~r, satisfying SLg_j.) < V2 — 1, Theorems 12. II 
and 12.21 guarantee that VAi satisfies the NSP of order s — r. Thus, given x = (a;i,a;2) such that xi E K^^*" is (s — r)— sparse 
and X2 € K'^, xi can be recovered by minimizing the ^i-norm of xi subject to {VAi)xi — VAx and, recalling that A2 is 
full-column rank, X2 — X2 is uniquely determined by A2X2 = y — Aixi. (In particular, this implies that A satisfies the NSP 
of order s — r for partially sparse recovery of size N — r.) 

IV. Partially sparse recovery implied by fully sparse recovery conditions 

We are now interested in showing that partially sparse recovery is achievable under the conditions which guarantee fully 
sparse recovery. In particular we will show that the NSP and RIP imply, respectively, the partial NSP and the partial RIP. We 
first establish the relationship between the corresponding null space properties. 

Theorem 4.1: If a given matrix A satisfies the NSP of order s then it satisfies the NSP for partially sparse recovery of order 
s — r for any r < s. 

Proof: Let A — {Ai,A2) satisfy the NSP of order s. First we note that since r < s, the NSP implies that A2 is full 
column rank. Let vi G M^^'' be a non-zero vector such that AiVi G TZ{A2) and let T e[N — r]'^*^'''^ 

Since there exists V2 such that Aivi + A2V2 — 0, we have that v = (ui,W2) G ■N'{A) \ {0}, and therefore by setting 
S = TU ([TV] \ [N-r]) and by using the NSP, \\{vi)t\\i + ||«2||i = ||«s||i < \\\v\\i = ^^ijli + ^||^2||i. Thus, ||(wi)t||i < 
II (wi)t II 1 + ^11^211 1 < 5 II Willi, and A satisfies the NSP of order s — r for partially sparse recovery of size N — r. ■ 

Partial RIP is also implied by RIP without the change in the RIP constant value. 

Theorem 4.2: Let 5s > and A = (^41,^2) satisfy the following property: For every (s — r)-sparse vector xi G M^^'" and 
X2 G M'' we have 

(1 - Ss)\\x\\l < \\Ax\\l < {1 + Ss)\\x\\l (8) 

where x = (xi, X2). Then A satisfies partial RIP of order s — r with 5l_^ — S for partially sparse recovery of size N — r, for 
any r < s. 

Proof: First we note that setting a;i = implies that A2 is full column rank. Consider now any given (s — r)— sparse 
vector xi G R^"*". Now, by setting X2 = - {Aj A2y Aj AiXi, one obtains (1 - (5s)||2;i||2 < (1 - ^s) {\\xi\\l + \\x2\\f) < 
\\AiXi + A2X2\\2 = WVAiXiW^. On the other hand, the choice X2 = provides ||7'Aia;i||2 < ||AiXi||| < (1 + Sg) HxiUj. We 
have thus arrived at the conditions of Definition 13.21 ■ 

Corollary 4.1: Let A = (Ai, A2) satisfy the RIP of order s with the RIP constant 5s. Then A satisfies partial RIP of order 
s — r with 5l_^ = Ss for partially sparse recovery of size N — r, for any r < s. 



V. Partial (and total) compressibility recovery with noisy measurements 

In most realistic applications the observed measurement vector y often contains noise and the true signal vector x is not 
sparse but rather compressible, meaning that most components are very small but not necessarily zero. It is known, however, 
that Compressed Sensing is robust to noise and can approximately recover compressible vectors. This statement is formalized 
in the following theorem taken from lITOl . 

Theorem 5.1: Assume that the matrix A S E''^^ satisfies RIP with the RIP constant 52s such that 52s < \/2— 1. For any 
X S M^, let noisy measurements y — Ax + e be given satisfying ||e||2 < rj. Let x'^ be a solution of 

min ||x||i s.t. \\Ax-y\\2<ri. (9) 



Then 

\\X*-X\\2 < CT] + d " "'-'1' \ (10) 



.#_^IL <r ..^W^iMi 



for constants c, d only depending on the RIP constant, and where as{x)i — min^. ||2:||o<s ll^; — x\\i. 

The following theorem provides an analogous result for the partially sparse recovery setting introduced in Section llll] 
Theorem 5.2: Assume that the matrix A — (Ai, A2) G M'''^^ satisfies partial RIP of order 2(s — r) for recovery of size 

N — r with the RIP constant 5^,_^-, < \/2 — 1. For any x — {xi, X2) G K^, let noisy measurements y = Ax + e be given 

satisfying ||e||2 < rj. Let x* — {xl,X2) be a solution of 



Then 



min llxilli s.t. \\Ax-y\\2<ri. (11) 

\K-x,\\2 < a^ + d^^^M^, (12) 

y's — r 



and 

lk;-X2||2 < C2(2r^ + cJcr^ + d^^^^^^Y (13) 

for constants c,d only depending on SL^_., and where Ci and C2 are given by Ci = ||y4i||2, and C2 = IIA2II2, (Since A2 
is full column rank recall that aI = {AjA2y^Aj and C2 > 0.) 

Proof: From Theorem 14.21 the matrix VAi, where V is given by d?), satisfies the condition of Theorem 15. II Thus, since 
T' is a projection matrix, \\VAiXi — Vy\\ = UPAS — VyW < \\Ax — y\\ < ij, and a solution xf of 

min llxilli s.t. \\VAiXi~Vy\\2<V, (14) 

xiGR"-'- 

satisfies 

||:.f-Si||2 < cv + d^^^M^. (15) 

^ys - r 

Now, we will prove that the solutions of problems (fTTT i and (fT4l i coincide in their xi parts, completing thus the proof of (fTzl i. 
Let {x\,X2) be a feasible point of (fTTT l. Again, since T-" is a projection matrix, we obtain that 

\\VA^xl-Vy\\2 = \\V{Aixl+A2x*2-y)\\2 

< \\Aixl + A2X*2 - y\\2 < V, 

which proves that x| is a feasible point of (fT4l i. Now let xf be a feasible point of (IT4l) . Since I — V projects (orthogonally) 
onto the column space of ^2 there must exist an x* such that A2X2 ~ {I — V){y — Aix*), and then ||Aia;f + ^20:2 — 2/II2 = 
IJT'Aix* —Vy\\2 < V- Therefore (xf , x*) is a feasible point of ( fTTT ). Hence we have proved that, any solution of problem (fTTT l 
is also a solution of problem (fT4l l. and the inequality (fT2] i results directly from (fTSt . 

We now use this inequality to bound the error on the reconstruction of X2. Since both x and x* satisfy the measurements 
constraints \\Ax~y\\2<rivis have that \\Ai{x\— xi) + A2{x2 — X2)\\2 < 2?7, and thus ||A2(a;2 — ^2)112 < 2?7+|jAi(xi — xi)||2. 
Using the definitions of Ci and C2 we have \\x2 — S2II2 < C2 {"^r] + Ci||x^ — 2;i||2), and the result (fT3T l follows from bounding 
1 1 2^1 — 2^1 II 2 by (fT2] | in this last inequality. ■ 

The condition on the matrix A imposed in the previous theorem involved only its partial RIP constant. In the next proposition 
we describe how one can bound the constants Ci and C2 in terms of the RIP constant of A (the proof is simple and is omitted, 
see also ITSl ). 

Proposition 5.1: Consider the RIP constant 5s of order s of A = {Ai , A2) G M*^^^. The constants Ci and C2 of Theorem |5.2| 
satisfy Ci < y/l + 5s and C2 < -7^=5=- 
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VI. Matrices with Partial RIP 

In this section we investigate regimes of N, s, and k for which random Gaussian matrices satisfy partial RIP. Similar results 
can be obtained for other families of random matrices, like sub-Gaussian or Bernoulli matrices. 

Theorem 6.1: Let < 5 < 1 and r < s. Let A = (^1,^2) with Ai e Mkx(N-r) ^^^^ ^^ ^ j^fexr j^^^g independent 
Gaussian entries with variance 1/fc. Then, as long as 

A = (Ai,A2) satisfies partial RIP of order s — r with (5^_^ < 6 for partially sparse recovery of size N — r, with high 
probability. 

Proof: Given a particular sparsity pattern, the probability that dH does not hold is (see ifTTl Lemma 5.1]) 

< 2 (12/(5)" e l^~^r. 

There are {^Zr) — ( Trf"^) different sparsity patterns (see, e.g., ifTTI ). Let V denote the probability that A = (^1,^2) 
does not satisfy the partial RIP of order s — r with (5^_^ = S for partially sparse recovery of size N — r. For this to happen, 
((SJl has to fail for at least one sparsity pattern, setting P — jg — ^ and using a union bound 

r < e("-'^)'°s(^e)2/'l^yg-/Jfc 

< 2e((""'')'°s(^e)+5log(J#)-/3fc) 

'N -r \-'-'-''^ (Yl 



< 21 e. , ^ 

where the second to last inequality was obtained using ( fTSI l. It is easy to see that either {e{N — r)/{s — r)) ^" ^' or (-y) 
goes to zero polynomially with N, thus V < O (A^^'-'^^^) ■ 

Note that the condition (fT6] l can be asymptotically smaller than the one found in the classical case r — Q. If, e.g., s — r — 0(1) 
then (fT6| just requires k — 0{s + log(iV — r)) instead of the classical k = 0{s \og{N/s)). 

VII. Concluding Remarks 

In some applications of Compressed Sensing one may be interested in a sparse (or compressible) vector whose support is 
partially known in advance. In such a setting we show that one can consider the £1 -minimization of the part of the vector 
for which the support is not known. We have shown that such a sparse recovery can be then ensured under conditions that 
are potentially weaker than those assumed for the full approach. We have explored this feature to show that it is possible to 
guarantee partial sparse recovery (with Gaussian random matrices) for an order of measurements below the one necessary for 
general recovery. 
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